AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	1.20244	1.44
	2_DecisionTree	Decision Tree	logloss	0.663664	21.29
	3_Linear	Linear	logloss	0.621415	12.6
	4_Default_Xgboost	Xgboost	logloss	0.614364	24.85
	5_Default_NeuralNetwork	Neural Network	logloss	0.69787	3.45
	6_Default_RandomForest	Random Forest	logloss	0.624269	28.74
the best	Ensemble	Ensemble	logloss	0.596263	0.3

AutoML Performance

AutoML Performance Boxplot

Features Importance

features importance across models

Spearman Correlation of Models

models spearman correlation

Summary of 5_Default_NeuralNetwork

<< Go back

Neural Network

n_jobs: -1
dense_1_size: 32
dense_2_size: 16
learning_rate: 0.05
num_class: 4
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

2.6 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.6	0.601626	0.791667	0.585492	0.705645	0.644696	0.688582	0.69787
recall	0.759036	0.604082	0.92605	0.356467	0.705645	0.661409	0.705645	0.69787
f1-score	0.670213	0.602851	0.853602	0.443137	0.705645	0.642451	0.68685	0.69787
support	83	245	595	317	0.705645	1240	1240	0.69787

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	63	18	0	2
Labeled as Major	38	148	14	45
Labeled as Minor	2	9	551	33
Labeled as Moderate	2	71	131	113

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Linear	2
4_Default_Xgboost	3
6_Default_RandomForest	1

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.619565	0.611336	0.842857	0.601476	0.727419	0.668809	0.720459	0.596263
recall	0.686747	0.616327	0.892437	0.514196	0.727419	0.677427	0.727419	0.596263
f1-score	0.651429	0.613821	0.866939	0.554422	0.727419	0.671653	0.722609	0.596263
support	83	245	595	317	0.727419	1240	1240	0.596263

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	57	25	0	1
Labeled as Major	31	151	8	55
Labeled as Minor	2	10	531	52
Labeled as Moderate	2	61	91	163

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 6_Default_RandomForest

<< Go back

Random Forest

n_jobs: -1
criterion: gini
max_features: 0.9
min_samples_split: 30
max_depth: 4
eval_metric_name: logloss
num_class: 4
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

27.7 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.619048	0.677419	0.844302	0.570552	0.728226	0.67783	0.726269	0.624269
recall	0.783133	0.514286	0.884034	0.586751	0.728226	0.692051	0.728226	0.624269
f1-score	0.691489	0.584687	0.863711	0.578538	0.728226	0.679606	0.72415	0.624269
support	83	245	595	317	0.728226	1240	1240	0.624269

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	65	15	0	3
Labeled as Major	35	126	8	76
Labeled as Minor	3	5	526	61
Labeled as Moderate	2	40	89	186

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Extreme (Fold 1)

SHAP Dependence from fold 1

Dependence Major (Fold 1)

SHAP Dependence from fold 1

Dependence Minor (Fold 1)

SHAP Dependence from fold 1

Dependence Moderate (Fold 1)

SHAP Dependence from fold 1

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

SHAP worst decisions from Fold 1

Worst decisions for selected sample 2 (Fold 1)

SHAP worst decisions from Fold 1

Worst decisions for selected sample 3 (Fold 1)

SHAP worst decisions from Fold 1

Worst decisions for selected sample 4 (Fold 1)

SHAP worst decisions from Fold 1

Best decisions for selected sample 1 (Fold 1)

SHAP best decisions from Fold 1

Best decisions for selected sample 2 (Fold 1)

SHAP best decisions from Fold 1

Best decisions for selected sample 3 (Fold 1)

SHAP best decisions from Fold 1

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

n_jobs: -1
num_class: 4
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

0.8 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0	0	0.479839	0	0.479839	0.11996	0.230245	1.20244
recall	0	0	1	0	0.479839	0.25	0.479839	1.20244
f1-score	0	0	0.648501	0	0.479839	0.162125	0.311176	1.20244
support	83	245	595	317	0.479839	1240	1240	1.20244

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	0	0	83	0
Labeled as Major	0	0	245	0
Labeled as Minor	0	0	595	0
Labeled as Moderate	0	0	317	0

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 4_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

n_jobs: -1
objective: multi:softprob
eta: 0.075
max_depth: 6
min_child_weight: 1
subsample: 1.0
colsample_bytree: 1.0
eval_metric: mlogloss
num_class: 4
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

23.8 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.641304	0.608871	0.84984	0.616788	0.734677	0.679201	0.728692	0.614364
recall	0.710843	0.616327	0.894118	0.533123	0.734677	0.688603	0.734677	0.614364
f1-score	0.674286	0.612576	0.871417	0.571912	0.734677	0.682548	0.730513	0.614364
support	83	245	595	317	0.734677	1240	1240	0.614364

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	59	22	0	2
Labeled as Major	30	151	9	55
Labeled as Minor	1	14	532	48
Labeled as Moderate	2	61	85	169

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Extreme (Fold 1)

SHAP Dependence from fold 1

Dependence Major (Fold 1)

SHAP Dependence from fold 1

Dependence Minor (Fold 1)

SHAP Dependence from fold 1

Dependence Moderate (Fold 1)

SHAP Dependence from fold 1

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

SHAP worst decisions from Fold 1

Worst decisions for selected sample 2 (Fold 1)

SHAP worst decisions from Fold 1

Worst decisions for selected sample 3 (Fold 1)

SHAP worst decisions from Fold 1

Worst decisions for selected sample 4 (Fold 1)

SHAP worst decisions from Fold 1

Best decisions for selected sample 1 (Fold 1)

SHAP best decisions from Fold 1

Best decisions for selected sample 2 (Fold 1)

SHAP best decisions from Fold 1

Best decisions for selected sample 3 (Fold 1)

SHAP best decisions from Fold 1

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 3_Linear

<< Go back

Logistic Regression (Linear)

n_jobs: -1
num_class: 4
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

11.5 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.644444	0.634146	0.835165	0.610487	0.733065	0.681061	0.725244	0.621415
recall	0.698795	0.636735	0.894118	0.514196	0.733065	0.685961	0.733065	0.621415
f1-score	0.67052	0.635438	0.863636	0.558219	0.733065	0.681953	0.727544	0.621415
support	83	245	595	317	0.733065	1240	1240	0.621415

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	58	24	0	1
Labeled as Major	28	156	11	50
Labeled as Minor	2	8	532	53
Labeled as Moderate	2	58	94	163

Learning curves

Coefficients

Coefficients learner #1

	Extreme	Major	Minor	Moderate
intercept	-5.23067	0.564709	2.41912	2.24684
Health Service Area	-0.0153044	-0.00597284	0.0596087	-0.0383315
Hospital County	0.065615	0.0152761	-0.0533957	-0.0274955
Operating Certificate Number	-0.194199	-0.0495815	0.0961094	0.147671
Facility Id	0.0651606	0.0357358	-0.037161	-0.0637354
Facility Name	0.00567867	-0.0477554	0.0912282	-0.0491514
Age Group	1.29904	0.706508	-1.65491	-0.350633
Zip Code - 3 digits	0.119265	-0.0610193	-0.0140682	-0.0441779
Gender	0.134831	0.0387713	-0.14447	-0.0291325
Race	0.00685708	0.0322749	0.00921874	-0.0483507
Ethnicity	0.198227	-0.0717455	-0.00631799	-0.120164
Length of Stay	0.0140195	0.0528018	-0.13354	0.0667189
Type of Admission	-0.0551736	0.0790164	-0.026742	0.00289918
Patient Disposition	-0.135887	0.205957	-0.15148	0.0814094
CCS Diagnosis Code	-0.545843	-0.149552	0.583893	0.111502
CCS Diagnosis Description	-0.0559129	0.0325011	0.082418	-0.0590062
CCS Procedure Code	0.147576	-0.0510445	0.0190545	-0.115586
CCS Procedure Description	0.177202	-0.0398291	-0.0815798	-0.0557932
APR DRG Code	0.340604	0.0658981	-0.219479	-0.187022
APR DRG Description	-0.0426926	0.00446023	-0.0163889	0.0546213
APR MDC Code	-0.197989	-0.0413347	0.0854664	0.153857
APR MDC Description	0.0421558	-0.0997725	0.172004	-0.114387
APR Severity of Illness Code	2.30134	0.735951	-2.29463	-0.742654
APR Severity of Illness Description	-1.19816	0.185024	0.586724	0.426414
APR Medical Surgical Description	-0.217383	-0.1366	0.304566	0.0494168
Payment Typology 1	0.0725904	0.0752514	-0.13974	-0.00810184
Payment Typology 2	-0.0847263	0.0571119	0.0719938	-0.0443794
Payment Typology 3	0.00943732	-0.0284599	0.0734202	-0.0543976
Birth Weight	-0.442874	0.392776	-0.024185	0.0742826
Emergency Department Indicator	0.148377	0.0279708	-0.154198	-0.0221505
Total Charges	0.162107	0.212341	-0.279689	-0.0947599
Total Costs	0.054338	-0.0866831	0.00991487	0.0224303

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Extreme (Fold 1)

SHAP Dependence from fold 1

Dependence Major (Fold 1)

SHAP Dependence from fold 1

Dependence Minor (Fold 1)

SHAP Dependence from fold 1

Dependence Moderate (Fold 1)

SHAP Dependence from fold 1

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

SHAP worst decisions from Fold 1

Worst decisions for selected sample 2 (Fold 1)

SHAP worst decisions from Fold 1

Worst decisions for selected sample 3 (Fold 1)

SHAP worst decisions from Fold 1

Worst decisions for selected sample 4 (Fold 1)

SHAP worst decisions from Fold 1

Best decisions for selected sample 1 (Fold 1)

SHAP best decisions from Fold 1

Best decisions for selected sample 2 (Fold 1)

SHAP best decisions from Fold 1

Best decisions for selected sample 3 (Fold 1)

SHAP best decisions from Fold 1

Best decisions for selected sample 4 (Fold 1)

SHAP best decisions from Fold 1

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

n_jobs: -1
criterion: gini
max_depth: 3
num_class: 4
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

20.3 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.609091	0.589474	0.815988	0.664835	0.723387	0.669847	0.718743	0.663664
recall	0.807229	0.685714	0.909244	0.381703	0.723387	0.695973	0.723387	0.663664
f1-score	0.694301	0.633962	0.860095	0.48497	0.723387	0.668332	0.708419	0.663664
support	83	245	595	317	0.723387	1240	1240	0.663664

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	67	16	0	0
Labeled as Major	38	168	15	24
Labeled as Minor	3	14	541	37
Labeled as Moderate	2	87	107	121

Learning curves

Decision Tree

Tree #1

Rules

if (APR Severity of Illness Description > 1.5) and (Age Group <= 3.5) and (Age Group <= 2.5) then class: Minor (proba: 95.18%) | based on 975 samples

if (APR Severity of Illness Description <= 1.5) and (APR Severity of Illness Code <= 3.5) and (Age Group > 2.5) then class: Major (proba: 57.79%) | based on 879 samples

if (APR Severity of Illness Description > 1.5) and (Age Group <= 3.5) and (Age Group > 2.5) then class: Minor (proba: 74.3%) | based on 603 samples

if (APR Severity of Illness Description > 1.5) and (Age Group > 3.5) and (APR Severity of Illness Code > 1.5) then class: Moderate (proba: 57.85%) | based on 567 samples

if (APR Severity of Illness Description <= 1.5) and (APR Severity of Illness Code > 3.5) and (Patient Disposition > 4.0) then class: Extreme (proba: 56.22%) | based on 233 samples

if (APR Severity of Illness Description > 1.5) and (Age Group > 3.5) and (APR Severity of Illness Code <= 1.5) then class: Minor (proba: 64.91%) | based on 228 samples

if (APR Severity of Illness Description <= 1.5) and (APR Severity of Illness Code <= 3.5) and (Age Group <= 2.5) then class: Minor (proba: 46.95%) | based on 164 samples

if (APR Severity of Illness Description <= 1.5) and (APR Severity of Illness Code > 3.5) and (Patient Disposition <= 4.0) then class: Extreme (proba: 89.86%) | based on 69 samples